Visual recognition of multi-agent action

Thesis (Ph.D.)--Massachusetts Institute of Technology, School of Architecture and Planning, Program in Media Arts and Sciences, 1999.Includes bibliographical references (p. 167-184).Developing computer vision sensing systems that work robustly in everyday environments will require that the systems can recognize structured interaction between people and objects in the world. This document presents a new theory for the representation and recognition of coordinated multi-agent action from noisy perceptual data. The thesis of this work is as follows: highly structured, multi-agent action can be recognized from noisy perceptual data using visually grounded goal-based primitives and low-order temporal relationships that are integrated in a probabilistic framework. The theory is developed and evaluated by examining general characteristics of multi-agent action, analyzing tradeoffs involved when selecting a representation for multi-agent action recognition, and constructing a system to recognize multi-agent action for a real task from noisy data. The representation, which is motivated by work in model-based object recognition and probabilistic plan recognition, makes four principal assumptions: (1) the goals of individual agents are natural atomic representational units for specifying the temporal relationships between agents engaged in group activities, (2) a high-level description of temporal structure of the action using a small set of low-order temporal and logical constraints is adequate for representing the relationships between the agent goals for highly structured, multi-agent action recognition, (3) Bayesian networks provide a suitable mechanism for integrating multiple sources of uncertain visual perceptual feature evidence, and (4) an automatically generated Bayesian network can be used to combine uncertain temporal information and compute the likelihood that a set of object trajectory data is a particular multi-agent action. The recognition algorithm is tested using a database of American football play descriptions. A system is described that can recognize single-agent and multi-agent actions in this domain given noisy trajectories of object movements. The strengths and limitations of the recognition system are discussed and compared with other multi-agent recognition algorithms.by Stephen Sean Intille.Ph.D

Tracking using a local closed-world assumption : tracking in the football domain

Thesis (M.S.)--Massachusetts Institute of Technology, Program in Media Arts & Sciences, 1994.Includes bibliographical references (leaves 81-85).by Stephen Sean Intille.M.S

Data, Data Everywhere, and Still Too Hard to Link: Insights from User Interactions with Diabetes Apps

For those with chronic conditions, such as Type 1 diabetes, smartphone apps offer the promise of an affordable, convenient, and personalized disease management tool. How- ever, despite significant academic research and commercial development in this area, diabetes apps still show low adoption rates and underwhelming clinical outcomes. Through user-interaction sessions with 16 people with Type 1 diabetes, we provide evidence that commonly used interfaces for diabetes self-management apps, while providing certain benefits, can fail to explicitly address the cognitive and emotional requirements of users. From analysis of these sessions with eight such user interface designs, we report on user requirements, as well as interface benefits, limitations, and then discuss the implications of these findings. Finally, with the goal of improving these apps, we identify 3 questions for designers, and review for each in turn: current shortcomings, relevant approaches, exposed challenges, and potential solutions

Acquiring in situ training data for context-aware ubiquitous computing applications

Ubiquitous, context-aware computer systems may ultimately enable computer applications that naturally and usefully respond to a user's everyday activity. Although new algorithms that can automatically detect context from wearable and environmental sensor systems show promise, many of the most flexible and robust systems use probabilistic detection algorithms that require extensive libraries of training data with labeled examples. In this paper, we describe the need for such training data and some challenges we have identified when trying to collect it while testing three contextdetection systems for ubiquitous computing and mobile applications. Author Keywords Context-aware, ubiquitous, computing, supervised learning, experience sampling, user interface design ACM Classification Keywords H5.m Information interfaces and presentation (e.g. HCI): Miscellaneous

Talk2Care: Facilitating Asynchronous Patient-Provider Communication with Large-Language-Model

Despite the plethora of telehealth applications to assist home-based older adults and healthcare providers, basic messaging and phone calls are still the most common communication methods, which suffer from limited availability, information loss, and process inefficiencies. One promising solution to facilitate patient-provider communication is to leverage large language models (LLMs) with their powerful natural conversation and summarization capability. However, there is a limited understanding of LLMs' role during the communication. We first conducted two interview studies with both older adults (N=10) and healthcare providers (N=9) to understand their needs and opportunities for LLMs in patient-provider asynchronous communication. Based on the insights, we built an LLM-powered communication system, Talk2Care, and designed interactive components for both groups: (1) For older adults, we leveraged the convenience and accessibility of voice assistants (VAs) and built an LLM-powered VA interface for effective information collection. (2) For health providers, we built an LLM-based dashboard to summarize and present important health information based on older adults' conversations with the VA. We further conducted two user studies with older adults and providers to evaluate the usability of the system. The results showed that Talk2Care could facilitate the communication process, enrich the health information collected from older adults, and considerably save providers' efforts and time. We envision our work as an initial exploration of LLMs' capability in the intersection of healthcare and interpersonal communication.Comment: Under submission to CHI202

Designing for Diabetes Decision Support Systems with Fluid Contextual Reasoning

Type 1 diabetes is a potentially life-threatening chronic condition that requires frequent interactions with diverse data to inform treatment decisions. While mobile technolo- gies such as blood glucose meters have long been an essen- tial part of this process, designing interfaces that explicitly support decision-making remains challenging. Dual-process models are a common approach to understanding such cog- nitive tasks. However, evidence from the first of two stud- ies we present suggests that in demanding and complex situations, some individuals approach disease management in distinctive ways that do not seem to fit well within existing models. This finding motivated, and helped frame our second study, a survey (n=192) to investigate these behaviors in more detail. On the basis of the resulting analysis, we posit Fluid Contextual Reasoning to explain how some people with diabetes respond to particular situations, and discuss how an extended framework might help inform the design of user interfaces for diabetes management

Protocol for the 'e-Nudge trial' : a randomised controlled trial of electronic feedback to reduce the cardiovascular risk of individuals in general practice [ISRCTN64828380]

Background: Cardiovascular disease (including coronary heart disease and stroke) is a major cause of death and disability in the United Kingdom, and is to a large extent preventable, by lifestyle modification and drug therapy. The recent standardisation of electronic codes for cardiovascular risk variables through the United Kingdom's new General Practice contract provides an opportunity for the application of risk algorithms to identify high risk individuals. This randomised controlled trial will test the benefits of an automated system of alert messages and practice searches to identify those at highest risk of cardiovascular disease in primary care databases. Design: Patients over 50 years old in practice databases will be randomised to the intervention group that will receive the alert messages and searches, and a control group who will continue to receive usual care. In addition to those at high estimated risk, potentially high risk patients will be identified who have insufficient data to allow a risk estimate to be made. Further groups identified will be those with possible undiagnosed diabetes, based either on elevated past recorded blood glucose measurements, or an absence of recent blood glucose measurement in those with established cardiovascular disease. Outcome measures: The intervention will be applied for two years, and outcome data will be collected for a further year. The primary outcome measure will be the annual rate of cardiovascular events in the intervention and control arms of the study. Secondary measures include the proportion of patients at high estimated cardiovascular risk, the proportion of patients with missing data for a risk estimate, and the proportion with undefined diabetes status at the end of the trial

Technological Innovations Enabling Automatic, Context-Sensitive Ecological Momentary Assessment

Health-related behavior, subjective states, cognitions, and interpersonal experiences are inextricably linked to context. Context includes information about location, time, past activities, interaction with other people and objects, and mental, physiological, and emotional states. Most real-time data collection methodologies require that subjects self-report information about contextual influences, notwithstanding the difficulty they have identifying the contextual factors that are influencing their behavior and subjective states. Often these assessment methodologies ask subjects to report on their activities or thoughts long after the actual events, thereby relying on retrospective recall and introducing memory biases. The “gold standard” alternative to these self-report instruments is direct observation. Direct observation in a laboratory setting, however, artificially constrains behavior. Direct observation is also typically too costly and invasive for long-term, large-sample-size studies of people in their natural environments.National Science Foundation (U.S.) (NSF ITR grant #0112900)Massachusetts Institute of Technology (House_n Consortium